本文提供了有关在机器学习(ML)模型中实现公平性的偏置缓解方法的全面调查。我们总共收集了234个有关ML分类器偏置缓解的出版物。这些方法可以根据其干预程序(即预处理,进行内部处理,后处理)及其应用的技术来区分。我们研究了文献中如何评估现有的缓解方法。特别是,我们考虑数据集,指标和基准测试。根据收集的见解(例如,最受欢迎的公平度量是什么?用于评估偏置缓解方法的数据集?)。我们希望在开发和评估新的缓解方法时支持从业者做出明智的选择。
translated by 谷歌翻译
软件偏见是软件工程师越来越重要的操作问题。我们提出了17种代表性缓解方法的大规模,全面的经验评估,该方法通过1​​2个机器学习(ML)绩效指标,4项公平度量指标和24种类型的公平性 - 性能权衡评估,应用于8种广泛采用的公平性折衷评估基准软件决策/预测任务。与以前在此重要的操作软件特征上的工作相比,经验覆盖范围是全面的,涵盖了最多的偏见缓解方法,评估指标和公平性的绩效权衡措施。我们发现(1)偏置缓解方法大大降低了所有ML性能指标(包括先前工作中未考虑的指标)所报告的值,在很大一部分的情况下(根据不同的ML性能指标为42%〜75%) ; (2)在所有情况和指标中,偏置缓解方法仅在约50%的情况下获得公平性改善(根据用于评估偏见/公平性的指标,介于29%〜59%之间); (3)缓解偏见的方法的表现不佳,甚至导致37%的情况下的公平性和ML性能下降; (4)缓解偏差方法的有效性取决于任务,模型,公平性和ML性能指标,并且没有证明对所有研究的情况有效的“银弹”缓解方法。在仅29%的方案中,我们发现优于其他方法的最佳缓解方法。我们已公开提供本研究中使用的脚本和数据,以便将来复制和扩展我们的工作。
translated by 谷歌翻译
图形卷积神经网络(GCN)吸引了越来越多的注意力,并在各种计算机视觉任务中取得了良好的表现,但是,对GCN的内部机制缺乏明确的解释。对于标准的卷积神经网络(CNN),通常使用类激活映射(CAM)方法通过生成热图来可视化CNN的决策和图像区域之间的连接。尽管如此,当这些凸轮直接应用于GCN时,这种热图通常会显示出语义 - chaos。在本文中,我们提出了一种新颖的可视化方法,特别适用于GCN,顶点语义类激活映射(VS-CAM)。 VS-CAM包括两个独立的管道,分别制作一组语义探针图和一个语义基映射。语义探针图用于检测语义信息从语义碱图图中的语义信息,以汇总语义感知的热图。定性结果表明,VS-CAM可以获得与基于CNN的CAM更精确地匹配对象的热图。定量评估进一步证明了VS-CAM的优势。
translated by 谷歌翻译
我们提出了地形遍历映射(TTM),是一个非结构化环境中自主挖掘机的地形推动性估算和路径规划的实时映射方法。我们提出了一种高效的基于学习的几何方法,可以从RGB图像和3D PointClouds中提取地形特征,并将它们纳入全球地图以进行自主挖掘的规划和导航。我们的方法使用了挖掘机的物理特性,包括最大攀爬程度和其他机器规格,以确定可遍历的区域。我们的方法可以适应更改环境并实时更新地形信息。此外,我们准备了一个小说数据集,自主挖掘机地形(AET)数据集,由来自施工站点的RGB图像,根据导航性,七个类别。我们将我们的映射方法与自动挖掘机导航系统中的规划和控制模块集成在一起,这在基于现有规划计划的成功率方面优于前面的方法49.3%。通过我们的映射,挖掘机可以通过由深坑,陡峭的山丘,岩石桩和其他复杂地形特征的非结构化环境导航。
translated by 谷歌翻译
Deep learning models can achieve high accuracy when trained on large amounts of labeled data. However, real-world scenarios often involve several challenges: Training data may become available in installments, may originate from multiple different domains, and may not contain labels for training. Certain settings, for instance medical applications, often involve further restrictions that prohibit retention of previously seen data due to privacy regulations. In this work, to address such challenges, we study unsupervised segmentation in continual learning scenarios that involve domain shift. To that end, we introduce GarDA (Generative Appearance Replay for continual Domain Adaptation), a generative-replay based approach that can adapt a segmentation model sequentially to new domains with unlabeled data. In contrast to single-step unsupervised domain adaptation (UDA), continual adaptation to a sequence of domains enables leveraging and consolidation of information from multiple domains. Unlike previous approaches in incremental UDA, our method does not require access to previously seen data, making it applicable in many practical scenarios. We evaluate GarDA on two datasets with different organs and modalities, where it substantially outperforms existing techniques.
translated by 谷歌翻译
The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.
translated by 谷歌翻译
As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.
translated by 谷歌翻译
Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.
translated by 谷歌翻译
We propose a distributionally robust return-risk model for Markov decision processes (MDPs) under risk and reward ambiguity. The proposed model optimizes the weighted average of mean and percentile performances, and it covers the distributionally robust MDPs and the distributionally robust chance-constrained MDPs (both under reward ambiguity) as special cases. By considering that the unknown reward distribution lies in a Wasserstein ambiguity set, we derive the tractable reformulation for our model. In particular, we show that that the return-risk model can also account for risk from uncertain transition kernel when one only seeks deterministic policies, and that a distributionally robust MDP under the percentile criterion can be reformulated as its nominal counterpart at an adjusted risk level. A scalable first-order algorithm is designed to solve large-scale problems, and we demonstrate the advantages of our proposed model and algorithm through numerical experiments.
translated by 谷歌翻译
Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.
translated by 谷歌翻译